WriteTaskStatsTracker

WriteTaskStatsTracker is the abstraction of WriteTaskStatsTrackers that collect the statistics of the number of buckets, files, partitions and rows processed.

Table 1. WriteTaskStatsTracker Contract
Method Description

getFinalStats

getFinalStats(): WriteTaskStats

The final WriteTaskStats statistics computed so far

Used when EmptyDirectoryWriteTask, SingleDirectoryWriteTask and DynamicPartitionWriteTask are requested to execute

newBucket

newBucket(bucketId: Int): Unit

Used when…​FIXME

newFile

newFile(filePath: String): Unit

Used when…​FIXME

newPartition

newPartition(partitionValues: InternalRow): Unit

Used when…​FIXME

newRow

newRow(row: InternalRow): Unit

Used when…​FIXME

Note
BasicWriteTaskStatsTracker is the one and only known implementation of the WriteTaskStatsTracker Contract in Apache Spark.

results matching ""

    No results matching ""