StateManagerImplBase

StateManagerImplBase is the extension of the StateManager contract for state managers of FlatMapGroupsWithStateExec physical operator with the following features:

Use Catalyst expressions for state serialization and deserialization
Use timeoutTimestampOrdinalInRow when shouldStoreTimestamp with the shouldStoreTimestamp flag on

Table 1. StateManagerImplBase Contract (Abstract Methods Only)
Method	Description
`stateDeserializerExpr`	`stateDeserializerExpr: Expression` State deserializer, i.e. a Catalyst expression to deserialize a state object from a row (`UnsafeRow`) Used exclusively for the stateDeserializerFunc
`stateSerializerExprs`	`stateSerializerExprs: Seq[Expression]` State serializer, i.e. Catalyst expressions to serialize a state object to a row (`UnsafeRow`) Used exclusively for the stateSerializerFunc
`timeoutTimestampOrdinalInRow`	`timeoutTimestampOrdinalInRow: Int` Position of the timeout timestamp in a state row Used when `StateManagerImplBase` is requested to get and set timeout timestamp

Table 2. StateManagerImplBases
StateManagerImplBase	Description
StateManagerImplV1	Legacy StateManager
StateManagerImplV2	Default StateManager

Creating StateManagerImplBase Instance

StateManagerImplBase takes a single shouldStoreTimestamp flag to be created (that is set when the concrete StateManagerImplBases are created).

Note	`StateManagerImplBase` is a Scala abstract class and cannot be created directly. It is created indirectly for the concrete StateManagerImplBases.

StateManagerImplBase initializes the internal properties.

Getting State Data for Key from StateStore — `getState` Method

getState(
  store: StateStore,
  keyRow: UnsafeRow): StateData

Note	`getState` is part of the StateManager Contract to get the state data for the key from the StateStore.

getState…FIXME

Persisting State Value for Key in StateStore — `putState` Method

putState(
  store: StateStore,
  key: UnsafeRow,
  state: Any,
  timestamp: Long): Unit

Note	`putState` is part of the StateManager Contract to persist (put) the state value for the key in the StateStore.

putState…FIXME

Removing State for Key from StateStore — `removeState` Method

removeState(
  store: StateStore,
  keyRow: UnsafeRow): Unit

Note	`removeState` is part of the StateManager Contract to remove the state for the key from the StateStore.

removeState…FIXME

Getting All State Data (for All Keys) from StateStore — `getAllState` Method

getAllState(store: StateStore): Iterator[StateData]

Note	`getAllState` is part of the StateManager Contract to retrieve all state data (for all keys) from the StateStore.

getAllState…FIXME

`getStateObject` Internal Method

getStateObject(row: UnsafeRow): Any

getStateObject…FIXME

Note	`getStateObject` is used when…FIXME

`getStateRow` Internal Method

getStateRow(obj: Any): UnsafeRow

getStateRow…FIXME

Note	`getStateRow` is used when…FIXME

Getting Timeout Timestamp (from State Row) — `getTimestamp` Internal Method

getTimestamp(stateRow: UnsafeRow): Long

getTimestamp…FIXME

Note	`getTimestamp` is used when…FIXME

Setting Timeout Timestamp (to State Row) — `setTimestamp` Internal Method

setTimestamp(
  stateRow: UnsafeRow,
  timeoutTimestamps: Long): Unit

setTimestamp…FIXME

Note	`setTimestamp` is used when…FIXME

Internal Properties

Name Description

Name	Description
`stateSerializerFunc`	State object serializer (of type `Any ⇒ UnsafeRow`) to serialize a state object (for a per-group state key) to a row (`UnsafeRow`) The serialization expression (incl. the type) is specified as the stateSerializerExprs Used exclusively in getStateRow
`stateDeserializerFunc`	State object deserializer (of type `InternalRow ⇒ Any`) to deserialize a row (for a per-group state value) to a Scala value The deserialization expression (incl. the type) is specified as the stateDeserializerExpr Used exclusively in getStateObject
`stateDataForGets`	Empty `StateData` to share (reuse) between getState calls (to avoid high use of memory with many `StateData` objects)

stateSerializerFunc

State object serializer (of type Any ⇒ UnsafeRow) to serialize a state object (for a per-group state key) to a row (UnsafeRow)

The serialization expression (incl. the type) is specified as the stateSerializerExprs

Used exclusively in getStateRow

stateDeserializerFunc

State object deserializer (of type InternalRow ⇒ Any) to deserialize a row (for a per-group state value) to a Scala value

The deserialization expression (incl. the type) is specified as the stateDeserializerExpr

Used exclusively in getStateObject

stateDataForGets

Empty StateData to share (reuse) between getState calls (to avoid high use of memory with many StateData objects)

StateManagerImplBase