Package edu.wpi.first.util.datalog

Class DataLog

java.lang.Object
edu.wpi.first.util.datalog.DataLog
All Implemented Interfaces:
AutoCloseable
public final class DataLog
extends Object
implements AutoCloseable
A data log. The log file is created immediately upon construction with a temporary filename. The file may be renamed at any time using the setFilename() function.

The data log is periodically flushed to disk. It can also be explicitly flushed to disk by using the flush() function.

The finish() function is needed only to indicate in the log that a particular entry is no longer being used (it releases the name to ID mapping). The finish() function is not required to be called for data to be flushed to disk; entries in the log are written as append() calls are being made. In fact, finish() does not need to be called at all.

DataLog calls are thread safe. DataLog uses a typical multiple-supplier, single-consumer setup. Writes to the log are atomic, but there is no guaranteed order in the log when multiple threads are writing to it; whichever thread grabs the write mutex first will get written first. For this reason (as well as the fact that timestamps can be set to arbitrary values), records in the log are not guaranteed to be sorted by timestamp.

  • Constructor Summary

    Constructors 
    Constructor Description
    DataLog()
    Construct a new Data Log.
    DataLog​(String dir)
    Construct a new Data Log.
    DataLog​(String dir, String filename)
    Construct a new Data Log.
    DataLog​(String dir, String filename, double period)
    Construct a new Data Log.
    DataLog​(String dir, String filename, double period, String extraHeader)
    Construct a new Data Log.

  • Method Summary

    Modifier and Type Method Description
    void addSchema​(Protobuf<?,​?> proto)
    Registers a protobuf schema.
    void addSchema​(Protobuf<?,​?> proto, long timestamp)
    Registers a protobuf schema.
    void addSchema​(Struct<?> struct)
    Registers a struct schema.
    void addSchema​(Struct<?> struct, long timestamp)
    Registers a struct schema.
    void addSchema​(String name, String type, byte[] schema)
    Registers a data schema.
    void addSchema​(String name, String type, byte[] schema, long timestamp)
    Registers a data schema.
    void addSchema​(String name, String type, String schema)
    Registers a data schema.
    void addSchema​(String name, String type, String schema, long timestamp)
    Registers a data schema.
    void appendBoolean​(int entry, boolean value, long timestamp)
    Appends a boolean record to the log.
    void appendBooleanArray​(int entry, boolean[] arr, long timestamp)
    Appends a boolean array record to the log.
    void appendDouble​(int entry, double value, long timestamp)
    Appends a double record to the log.
    void appendDoubleArray​(int entry, double[] arr, long timestamp)
    Appends a double array record to the log.
    void appendFloat​(int entry, float value, long timestamp)
    Appends a float record to the log.
    void appendFloatArray​(int entry, float[] arr, long timestamp)
    Appends a float array record to the log.
    void appendInteger​(int entry, long value, long timestamp)
    Appends an integer record to the log.
    void appendIntegerArray​(int entry, long[] arr, long timestamp)
    Appends an integer array record to the log.
    void appendRaw​(int entry, byte[] data, int start, int len, long timestamp)
    Appends a record to the log.
    void appendRaw​(int entry, byte[] data, long timestamp)
    Appends a raw record to the log.
    void appendRaw​(int entry, ByteBuffer data, int start, int len, long timestamp)
    Appends a record to the log.
    void appendRaw​(int entry, ByteBuffer data, long timestamp)
    Appends a record to the log.
    void appendString​(int entry, String value, long timestamp)
    Appends a string record to the log.
    void appendStringArray​(int entry, String[] arr, long timestamp)
    Appends a string array record to the log.
    void close()  
    void finish​(int entry)
    Finish an entry.
    void finish​(int entry, long timestamp)
    Finish an entry.
    void flush()
    Explicitly flushes the log data to disk.
    long getImpl()  
    boolean hasSchema​(String name)
    Returns whether there is a data schema already registered with the given name.
    void pause()
    Pauses appending of data records to the log.
    void resume()
    Resumes appending of data records to the log.
    void setFilename​(String filename)
    Change log filename.
    void setMetadata​(int entry, String metadata)
    Updates the metadata for an entry.
    void setMetadata​(int entry, String metadata, long timestamp)
    Updates the metadata for an entry.
    int start​(String name, String type)
    Start an entry.
    int start​(String name, String type, String metadata)
    Start an entry.
    int start​(String name, String type, String metadata, long timestamp)
    Start an entry.
    void stop()
    Stops appending all records to the log, and closes the log file.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • DataLog

      public DataLog​(String dir, String filename, double period, String extraHeader)
      Construct a new Data Log. The log will be initially created with a temporary filename.
      Parameters:
      dir - directory to store the log
      filename - filename to use; if none provided, a random filename is generated of the form "wpilog_{}.wpilog"
      period - time between automatic flushes to disk, in seconds; this is a time/storage tradeoff
      extraHeader - extra header data

    • DataLog

      public DataLog​(String dir, String filename, double period)
      Construct a new Data Log. The log will be initially created with a temporary filename.
      Parameters:
      dir - directory to store the log
      filename - filename to use; if none provided, a random filename is generated of the form "wpilog_{}.wpilog"
      period - time between automatic flushes to disk, in seconds; this is a time/storage tradeoff

    • DataLog

      public DataLog​(String dir, String filename)
      Construct a new Data Log. The log will be initially created with a temporary filename.
      Parameters:
      dir - directory to store the log
      filename - filename to use; if none provided, a random filename is generated of the form "wpilog_{}.wpilog"

    • DataLog

      public DataLog​(String dir)
      Construct a new Data Log. The log will be initially created with a temporary filename.
      Parameters:
      dir - directory to store the log

    • DataLog

      public DataLog()
      Construct a new Data Log. The log will be initially created with a temporary filename.

  • Method Details

    • setFilename

      public void setFilename​(String filename)
      Change log filename.
      Parameters:
      filename - filename

    • flush

      public void flush()
      Explicitly flushes the log data to disk.

    • pause

      public void pause()
      Pauses appending of data records to the log. While paused, no data records are saved (e.g. AppendX is a no-op). Has no effect on entry starts / finishes / metadata changes.

    • resume

      public void resume()
      Resumes appending of data records to the log. If called after stop(), opens a new file (with random name if SetFilename was not called after stop()) and appends Start records and schema data values for all previously started entries and schemas.

    • stop

      public void stop()
      Stops appending all records to the log, and closes the log file.

    • hasSchema

      public boolean hasSchema​(String name)
      Returns whether there is a data schema already registered with the given name.
      Parameters:
      name - Name (the string passed as the data type for records using this schema)
      Returns:
      True if schema already registered

    • addSchema

      public void addSchema​(String name, String type, byte[] schema, long timestamp)
      Registers a data schema. Data schemas provide information for how a certain data type string can be decoded. The type string of a data schema indicates the type of the schema itself (e.g. "protobuf" for protobuf schemas, "struct" for struct schemas, etc). In the data log, schemas are saved just like normal records, with the name being generated from the provided name: "/.schema/name". Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      name - Name (the string passed as the data type for records using this schema)
      type - Type of schema (e.g. "protobuf", "struct", etc)
      schema - Schema data
      timestamp - Time stamp (may be 0 to indicate now)

    • addSchema

      public void addSchema​(String name, String type, byte[] schema)
      Registers a data schema. Data schemas provide information for how a certain data type string can be decoded. The type string of a data schema indicates the type of the schema itself (e.g. "protobuf" for protobuf schemas, "struct" for struct schemas, etc). In the data log, schemas are saved just like normal records, with the name being generated from the provided name: "/.schema/name". Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      name - Name (the string passed as the data type for records using this schema)
      type - Type of schema (e.g. "protobuf", "struct", etc)
      schema - Schema data

    • addSchema

      public void addSchema​(String name, String type, String schema, long timestamp)
      Registers a data schema. Data schemas provide information for how a certain data type string can be decoded. The type string of a data schema indicates the type of the schema itself (e.g. "protobuf" for protobuf schemas, "struct" for struct schemas, etc). In the data log, schemas are saved just like normal records, with the name being generated from the provided name: "/.schema/name". Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      name - Name (the string passed as the data type for records using this schema)
      type - Type of schema (e.g. "protobuf", "struct", etc)
      schema - Schema data
      timestamp - Time stamp (may be 0 to indicate now)

    • addSchema

      public void addSchema​(String name, String type, String schema)
      Registers a data schema. Data schemas provide information for how a certain data type string can be decoded. The type string of a data schema indicates the type of the schema itself (e.g. "protobuf" for protobuf schemas, "struct" for struct schemas, etc). In the data log, schemas are saved just like normal records, with the name being generated from the provided name: "/.schema/name". Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      name - Name (the string passed as the data type for records using this schema)
      type - Type of schema (e.g. "protobuf", "struct", etc)
      schema - Schema data

    • addSchema

      public void addSchema​(Protobuf<?,​?> proto, long timestamp)
      Registers a protobuf schema. Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      proto - protobuf serialization object
      timestamp - Time stamp (0 to indicate now)

    • addSchema

      public void addSchema​(Protobuf<?,​?> proto)
      Registers a protobuf schema. Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      proto - protobuf serialization object

    • addSchema

      public void addSchema​(Struct<?> struct, long timestamp)
      Registers a struct schema. Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      struct - struct serialization object
      timestamp - Time stamp (0 to indicate now)

    • addSchema

      public void addSchema​(Struct<?> struct)
      Registers a struct schema. Duplicate calls to this function with the same name are silently ignored.
      Parameters:
      struct - struct serialization object

    • start

      public int start​(String name, String type, String metadata, long timestamp)
      Start an entry. Duplicate names are allowed (with the same type), and result in the same index being returned (start/finish are reference counted). A duplicate name with a different type will result in an error message being printed to the console and 0 being returned (which will be ignored by the append functions).
      Parameters:
      name - Name
      type - Data type
      metadata - Initial metadata (e.g. data properties)
      timestamp - Time stamp (0 to indicate now)
      Returns:
      Entry index

    • start

      public int start​(String name, String type, String metadata)
      Start an entry. Duplicate names are allowed (with the same type), and result in the same index being returned (start/finish are reference counted). A duplicate name with a different type will result in an error message being printed to the console and 0 being returned (which will be ignored by the append functions).
      Parameters:
      name - Name
      type - Data type
      metadata - Initial metadata (e.g. data properties)
      Returns:
      Entry index

    • start

      public int start​(String name, String type)
      Start an entry. Duplicate names are allowed (with the same type), and result in the same index being returned (start/finish are reference counted). A duplicate name with a different type will result in an error message being printed to the console and 0 being returned (which will be ignored by the append functions).
      Parameters:
      name - Name
      type - Data type
      Returns:
      Entry index

    • finish

      public void finish​(int entry, long timestamp)
      Finish an entry.
      Parameters:
      entry - Entry index
      timestamp - Time stamp (0 to indicate now)

    • finish

      public void finish​(int entry)
      Finish an entry.
      Parameters:
      entry - Entry index

    • setMetadata

      public void setMetadata​(int entry, String metadata, long timestamp)
      Updates the metadata for an entry.
      Parameters:
      entry - Entry index
      metadata - New metadata for the entry
      timestamp - Time stamp (0 to indicate now)

    • setMetadata

      public void setMetadata​(int entry, String metadata)
      Updates the metadata for an entry.
      Parameters:
      entry - Entry index
      metadata - New metadata for the entry

    • appendRaw

      public void appendRaw​(int entry, byte[] data, long timestamp)
      Appends a raw record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      data - Byte array to record; will send entire array contents
      timestamp - Time stamp (0 to indicate now)

    • appendRaw

      public void appendRaw​(int entry, byte[] data, int start, int len, long timestamp)
      Appends a record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      data - Byte array to record
      start - Start position of data (in byte array)
      len - Length of data (must be less than or equal to data.length - start)
      timestamp - Time stamp (0 to indicate now)

    • appendRaw

      public void appendRaw​(int entry, ByteBuffer data, long timestamp)
      Appends a record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      data - Buffer to record; will send from data.position() to data.limit()
      timestamp - Time stamp (0 to indicate now)

    • appendRaw

      public void appendRaw​(int entry, ByteBuffer data, int start, int len, long timestamp)
      Appends a record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      data - Buffer to record
      start - Start position of data (in buffer)
      len - Length of data (must be less than or equal to data.capacity() - start)
      timestamp - Time stamp (0 to indicate now)

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable

    • appendBoolean

      public void appendBoolean​(int entry, boolean value, long timestamp)
      Appends a boolean record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      value - Boolean value to record
      timestamp - Time stamp (0 to indicate now)

    • appendInteger

      public void appendInteger​(int entry, long value, long timestamp)
      Appends an integer record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      value - Integer value to record
      timestamp - Time stamp (0 to indicate now)

    • appendFloat

      public void appendFloat​(int entry, float value, long timestamp)
      Appends a float record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      value - Float value to record
      timestamp - Time stamp (0 to indicate now)

    • appendDouble

      public void appendDouble​(int entry, double value, long timestamp)
      Appends a double record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      value - Double value to record
      timestamp - Time stamp (0 to indicate now)

    • appendString

      public void appendString​(int entry, String value, long timestamp)
      Appends a string record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      value - String value to record
      timestamp - Time stamp (0 to indicate now)

    • appendBooleanArray

      public void appendBooleanArray​(int entry, boolean[] arr, long timestamp)
      Appends a boolean array record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      arr - Boolean array to record
      timestamp - Time stamp (0 to indicate now)

    • appendIntegerArray

      public void appendIntegerArray​(int entry, long[] arr, long timestamp)
      Appends an integer array record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      arr - Integer array to record
      timestamp - Time stamp (0 to indicate now)

    • appendFloatArray

      public void appendFloatArray​(int entry, float[] arr, long timestamp)
      Appends a float array record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      arr - Float array to record
      timestamp - Time stamp (0 to indicate now)

    • appendDoubleArray

      public void appendDoubleArray​(int entry, double[] arr, long timestamp)
      Appends a double array record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      arr - Double array to record
      timestamp - Time stamp (0 to indicate now)

    • appendStringArray

      public void appendStringArray​(int entry, String[] arr, long timestamp)
      Appends a string array record to the log.
      Parameters:
      entry - Entry index, as returned by start()
      arr - String array to record
      timestamp - Time stamp (0 to indicate now)

    • getImpl

      public long getImpl()